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ABSTRACT 

Education in contemporary society is no longer the 
selective enterprise for which a product evaluation approach is 
appropriate* Process evaluation, however, complements a new view of 
education — education offers all students opportunities for individual 
growth* A process approach focuses on what is understood* Focusing on 
learning activi'^'v and targeting behavior, it accounts for the 
interaction of knowledge, skills, and perceptions as they emerge in a 
learning process* Opponents of process evaluation believe it is in 
conflict with the structure of schools, but research does not support 
this idea* Process evaluation is also often thought to lack 
standards, but, in fact, its standards are grounded in cognitive 
psychology and the concepts of systems testing. A final argument 
against process evaluation has been that it conflicts with the 
traditional aims of education* Measurement, however, must evolve to 
suit today’s education, which promises the development of individual 
skills. While many recognize the value of process evaluation, its 
application to large-scale programs will require sound research, 
collaboration between research and practice, the implementation of 
pilot programs, and national networking for information 
dissemination* (Contains AA references.) (SLD) 
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Instruction and evaluation in educational systems should 
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constitute an integral event (Glaser, 1981). This principle is 
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realized in process-oriented classes where the monitoring of 



students’ understanding of their skills guides the curriculum 
(Atwell, 1987; Brandt, 1990). In the last decade, many 
educators have altered classroom instruction in this manner, 
applying the insights of cognitive research. The techniques 
they practice, however, conflict with the external testing 
institutions typically employ to measure information acquired or 
general abilities and to compare student performance across 
time, districts, and the nation. On the surface, process 
assessment appears inadequate for stimulating such discussions 
about students, particularly when diversity in education is 
increasing. Education in our contemporary society, however, is 
no longer the selective enterprise for which a product approach 
is appropriate. Today, less emphasis is placed on selecting 
individuals for available educational opportunities, and more 
attention is devoted to offering all students individual 
opportunities for growth (Glaser, 1981). Process evaluation 
complements this new view of education. In this paper, I 
address the issues surrounding this approach to evaluation. 
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argue against common misconceptions which impede implementation, 
and consider what might be done to facilitate large-scale 
process testing • 

THE ADVANTAGES OF PROCESS EVALUATION 
AS IT OPPOSES PRODUCT MEASUREMENT 

The controversy surrounding assessment in education reflects 
the fundamental debate over product and process approaches to 
teaching (Dewey, 1938). In a product-oriented environment, 
instruction focuses on the reproduction of information, and 
evaluation measures general skills and the short term retention 
of isolated points. A process approach, in contrast, focuses on 
what is understood. The content of a particular domain is 
organized and structured within a meaningful frame of reference 
(Bowden, 1986). Evaluation in this setting consists of the 
ongoing monitoring of skills, perceptions, and performance by 
both teachers and students (Anstey and Ball, 1991). 

Product evaluation is a tradition in American education. For 
almost a century, it appeared both appropriate and efficient, 
given the enormity of compulsory education in America and its 
initial aim: the transmission of organized bodies of knowledge 

and prepared skills ( Cremin, 1988 ) . Within this f ramework , the 
focus of evaluation rests outside the classroom experience. 
Assessment measures the information and prepared skills 
targeted, most often through objective testing. This efficient 
practice accommodates the ranking of students , thus satisfying 
the desire to compare the outcomes of education across 
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individuals, schools, and districts on a nation-wide basis 
(Paris, Lawton, Turner, and Roth, 1991). Entrenched in 
America’s educational tradition, this orientation is, however, 
outdated. Grounded in education of the past, it assumes that 
cognition and learning can be separated from the act of 
acquiring and applying skills (Resnick and Resnick, 1989). 

Paris, Lawton, Turner, and Roth (1991) describe the 
cumulative effects of external testing. Confronted with 
external, decontextual ized assessment, experienced adolescent 
students are more likely to cheat, to become nervous, to have 
difficulty concentrating, and to guess than elementary students 
are. Low achievers, in particular, appear hostile and withdrawn 
by the adolescent years, manifesting the effects of almost a 
decade of testing in which they competed against others rather 
than competing with themselves. Haladyna, Nolen, and Haas 
(1991) add that external testing reifies learning processes and 
expresses a year’s worth, or even a career’s worth, of study in 
two or three digit numbers. These figures attribute 
achievements or failures influencing placement and progress to a 
single moment; to a single teacher, school, or district; and /or 
to a mere sampling or manipulation of the curriculum. 

Process evaluation reflects a different perspective on 
measuring student performance. Focusing on learning activity 
and targeting behavior, it accounts for the interaction of 
knowledge, skills, and perceptions as they emerge in a learning 
process (Nickerson, Perkins, and Smith, 1985). Evaluation 
involves the monitoring of skills. Monitoring fits learning as 
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it is shaped. It aims to intervene--to facilitate and direct 
developing skills and overall performance rather than to 
identify a need to "re-learn" something or to correct serious 
deviations in performance (Goodman, 1989). In classrooms, 
teachers introduce, explicitly, the knowledge and skills 
associated with learning in specific domains to stimulate 
monitoring activity (Siegler, 1986). They situate these skills 
in meaningful contexts to trigger student response (Perkins and 
Salomon, 1989). Visible in the materials and activities of the 
classroom are the essential components comprising the skills 
taught (Frederiksen and Collins, 1989). Framing learning 
experiences, these offer students clear reference points against 
which they can judge their own work and through which they can 
understand teachers’ judgments (Bowden, 1986). 

Monitoring is grounded in learning activities--opportuni ties 
to apply information. These activities include verbal 
activity, the act of answering questions and participating in 
discussions, debate, or role play; written work, the process of 
preparing texts which describe, explore, analyze, or critique 
the subject matter under study; and goal-directed actions, such 
as problem-solving, inquiry, the construction of models, 
displays, or portfolios, and oven formal testing (Brophy and 
Alleman, 1991). Rather than being evaluated with little sense 
of the purposes measurement serves, students develop an 
understanding of their capabilities and the utility of the 
information they study (Goodman, 1989). 
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The benefits of process-oriented learning experiences which 
integrate instruction and evaluation are many. They make the 
content studied and the skills employed both visible and 
intelligible. Linking course material with personal 
experiences, they trigger the internal processes which guide 
students to assess themselves (Glaser, 1987). The approach is 
significant, then, because it isolates and measures knowledge 
and skills directly rather than decontextualizing and 
manipulating them (Frederiksen and Collins, 1989). However, the 
unique focus this approach adopts offers an unfamiliar 
perspective on student performance. Student-centered and 
skills-oriented , process evaluation on the classroom level 
appears so idiosyncratic that both administrators and educators 
find its large-scale implementation difficult to envision. 

AN ARGUMENT AGAINST COMMON MISCONCEPTIONS 

Several factors explain why a movement to alter the focus of 
assessment has not yet occurred. First, misconceptions 
regarding process evaluation exist. Opponents often perceive it 
to be in conflict with the structure of the school day, 
standards, and the aims of education. These misconceptions 
point to a basic misunderstanding of the nature of process 
evaluation as it reflects research on cognition and contemporary 
views on education. 

The common misconception that process evaluation is in 
conflict with the structure of schools is grounded in the belief 
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that implementation will require the blurring of grade levels 
and content areas (Dewey, 1938). No support for these claims 
can be found in empirical studies of process learning, however. 
Process evaluation measures both information (declarative 
knowledge) and mental processes (procedural and meta knowledge) 
(Glaser, 1987). Research indicates that mental processes, 
though they may not be entirely stage-based, progress over the 
course of time (Damon, 1983). The levels typically associated 
with schooling, therefore, are not in conflict with learning 
programs which target developing skills. Additionally, current 
research on cognitive development (e.g., Brown, Collins, and 
Duguid, 1989; Siegler, 1986) indicates that mental processes are 
grounded in specific knowledge-rich contexts. A specified, 
focused knowledge base offers the foundation upon which skills 
can be exercised and knowledge from other areas triggered and 
integrated. The domains organizing formal education offer a 
framework for cognitive growth. This framework is most 
effective when it situates domain-specific knowledge and skills 
in authentic activity (Brown, Collins, and Duguid, 1989). 

Another misunderstanding regarding process evaluation is that 
it lacks standards (Nunan, 1988). Skeptics feel that parameters 
for discussing students on a large scale cannot exist in a 
setting which focuses on activity rather than external 
objectives. If process evaluation were implemented, therefore, 
society’s discussions of academic progress across students would 
no longer be possible. A careful examination of process 
evaluation indicates, however, that it is, indeed, governed by 
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standards. Accounting for the interaction of knowledge and 
skills, its standards are grounded in cognitive psychology 
rather than the organized bodies of knowledge constituting the 
various domains (Glaser, 1981). 

Theories on process testing (e.g., Frederiksen and Collins, 
1989) describe certain standards and components. Systems 
testing measures procedural and meta knowledge and attends to 
learners’ outcomes (David, 1991). It examines what individuals 
are capable of doing rather than identifying only those who will 
be successful performers (Glaser, 1981). To assess growth, 
careful analyses of learning activities in the disciplines are 
necessary (Frederiksen and Collins, 1989). These decompose 
complex skills into their component procedures and required 
knowledge, identifying the skills particular learning activities 
involve (Siegler, 1986). Aware of all possible knowledge, 
skills, and strategies required to perform well in targeted 
activities, both students and teachers can measure performance 
and understand ways to improve it (Frederiksen and Collins, 

1989). This type of evaluation achieves systemic validity; it 
measures directly that which is taught. While this approach 
focuses on individual performances across multiple learning 
activities, scoring systems for standardizing it are possible. 
The primary trait scoring system introduced in research on 
writing ( Lloyd- Jones , 1977) offers direction for their design. 

The primary traits of a specific writing task consist of the 
knowledge and skills necessary to complete the task. A "how-to" 
essay, for example, involves knowledge about the specific topic. 
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knowledge about the use of second person and temporal transition 
words, and the ability to sequence information logically. From 
these and other primary traits essential to the task, a scoring 
system and models of the task can be constructed. For primary 
trait scoring, the rater gives scores (points on a scale ranging 
from 1 to 6 , for example) based on criteria that have been 
established through task analysis. The criteria vary from one 
type of writing to another and are therefore rhetorically-based. 

From the detailed criteria organizing primary trait scoring 
in various domains, examplars of tasks can be constructed. 

These concrete, transparent, meaningful models of a task, as it 
may be executed over the course of time across individual, 
orient teachers to the criteria for judging student performance 
and students to the manner in which they will be evaluated. 
Depicting learning activity, examplars may consist of portfolios 
of written work or art work, video clips of presentations or 
projects, or a set of computer programs, for example. Offering 
the information which guides learning and evaluation and 
measuring skills directly, systems testing is superior to 
product evaluation, whose unspecified dimensions lack systemic 
validity (Frederiksen and Collins, 1989). 

The final claim against process evaluation is that it appears 
to be in conflict with the traditional aims of education. 

Without doubt, process evaluation opposes the purpose of early 
compulsory education in America--the transmission of fixed 
bodies of knowledge and prepared skills (Dewey, 1938). Over 
time, however, this goal has been altered to complement both a 
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rapidly changing society and insights on learning. Since 
today’s education promises the development of individual skills 
in a manner permitting ongoing adaptation in society, 
measurement must evolve through educational experiences rather 
than be imposed upon them (Chipman and Segal, 1985). 

Theoretical perspectives describe process evaluation, yet the 
a.pproach remains elusive because it focuses on abstract mental 
constructs. Integrated into individual classrooms and automatic 
to instructors who practice it, the specific features of process 
evaluation are often difficult to separate from the language of 
schools or psychology. While this type of evaluation becomes 
"wooly" in discussions of abstract cognitive functions, a 
familiar experience outside the field highlights some of its 
features . 

Certain standards exist for issuing licenses to new drivers. 
While manuals are read and written tests are taken, the final 
decision regarding a person’s driving skills rests with a 
process exam--the road test. An examiner rides with the new 
driver, monitoring his/her execution of certain essential tasks. 
The necessary skills include judgments and decisions--the 
application of knowledge about the rules of the road. 

Evaluators can assess a new driver’s skills because they have 
isolated the subskills and tasks which constitute good driving. 
The examiners observe the execution of required maneuvers listed 
on the scoring sheet, awarding points for subtasks performed 
successfully . 
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The preceding example exposes significant points about the 
large-scale implementation of process evaluation. First, the 
central issue surrounding evaluation does not appear to be its 
standardization but, rather, its orientation to product or 
recall and/or the manipulation of skills. This point suggests 
that the development of large-scale process assessment measures 
may not necessarily warrant a radical change in the form testing 
adopts. Perhaps test items which focus on dilemmas in 
meaningful contexts aligned with the curriculum can be written 
as objective questions. Whether process testing can adopt this 
traditional form while shifting its focus is a critical issue 
research exploring large-scale implementation must investigate. 
Second, while evaluation which targets skills and processes 
appears subject to the idiosyncrasies of teachers and students, 
the road test demonstrates that certain fundamental processes 
involving the use of domain-specific knowledge remain constant, 
despite individual variation. This insight also guides the 
development of suitable measures. 

TOWARD LARGE-SCALE PROCESS EVALUATION: ISSUES AND AGENDAS 



Process instruction has gained attention as a 
pedagogically-sound approach, whose focus is consistent with 
contemporary thought on the aims of education. However, the gap 
separating this type of learning from existing large-scale 
external evaluation is detrimental to students, teachers, 
schools. A movement to alter large-scale assessment is both 
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important and timely. Its success rests with a sound research 
agenda, the collaborative efforts of research and practice, 
pilot programs, and national networking. 

The research targeting large-scale implementation must 
describe, in detail, the qualitative progression of skills 
educational systems will measure over the course of time 
(Glaser, 1987). This information will be necessary to identify 
the performance levels of individuals in diverse settings and to 
direct their progress. Existing cognitively-oriented 
explanations of general skills development (e.g-, Sternberg, 

1977) will contribute to this awareness; they will not, however, 
provide all the information needed to develop a system for 
evaluating skills. 

An understanding of cognition, as it is situated within the 
experiences of individuals, various domains, and multiple 
settings for learning, is essential to the development of 
large-scale measurement procedures. Since contemporary research 
stresses the significance of knowledge and experience, with its 
rich descriptions of expertise in various domains (e.g., Chi, 
Feltovich, and Glaser, 1981; Flower and Hayes, 1986; Frederiksen 
and Collins, 1989), the manner in which the academic disciplines 
trigger thinking is a topic to explore. Contrasting the 
cognitive models which depict general skills, those which 
capture the interaction of skills and knowledge in specific 
domains across levels will be aligned with the curriculum of 
process learning. Within these interactive models, the 
problem-solving activity, sequential processes, and other skills 
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characterizing the procedural knowledge of certain domains may 
be detected. This information will offer insights on how to 
construct test items might be constructed and how to organize 
evaluation measures. The domain-specific models may also point 
to certain processes common to activities across domains that 
testing might target. In this cognitively-oriented research on 
evaluation, other questions or concerns follow logically. 

Whether testing must be individualized, whether language 
contributes to or confounds process evaluation , and the role 
technology might play in the "scanning" of sustained rather than 
objective responses to test items are state-of-the-art issues 
warranting investigation . 

Also included in the research agenda are efforts to 
understand the role product evaluation plays in learning. Since 
literature on cognitive development stresses the significance of 
domain-specific knowledge (Perkins and Salomon, 1989), the 
information students possess is essential to instruction. While 
product evaluation cannot dominate educational systems oriented 
to process (Glaser, 1981), information acquired is an essential 
element which cannot be neglected. Through the careful task 
analyses , pilot programs , and observations conducted in schools , 
an understanding of how product evaluation is integrated into 
process programs is likely to emerge. 

If the findings of this agenda are to have a clear impact on 
educational practice, then the agenda itself must be grounded in 
the collaborative efforts of research and practice. Only 
through an analysis of what occurs in process classrooms can one 
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begin to understand that which must be measured and the diverse 
populations who will be subject to measurement. When research 
is linked to the classroom, the procedures teachers intuitively 
adopt can be perceived, analyzed, and evaluated by research 
teams. They will learn, for example, whc^^her a process 
orientation to evaluation can be retained even in chort answer 
or objective questions. Also, through this collaborative 
effort, the individual differences testing must accommodate can 
be detected. This information will facilitate discussions about 
the form testing might take to minimize bias and to capitalize 
on individual differences while measuring performance against 
typical patterns of growth. In addition, the range of 
performances students might demonstrate are visible in the 
classroom, offering insights on how and why individuals adopt 
different approaches for completing similar tasks. 

The benefits of this collaborative investigation of skills 
will extend to educational practice itself in a manner 
benefiting the research effort. Involved in 
cognitively-oriented analyses of the tasks comprising the 
curriculum, teachers in various domains will understand better 
the nature of the skills they target. This information will 
enhance their ability to design suitable learning activities, to 
present them explicitly to students, and to monitor theii 
development. As research directs the organization of process 
learning, the resulting improvements in instruction will 
strengthen the overall effort to analyze the classroom 
experience from a cognitive perspective. 
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Small-scale pilot projects in the process classroom will 
provide the framework through which appropriate measures will be 
discovered. Directly involved with learning experiences, 
project directors will observe and analyze the authentic 
problems and dilemmas testing must parallel. They will 
administer multiple representations of seemingly appropriate 
test items in the classroom, formally and informally, to test 
their validity and to explore issues regarding the 
standardization. The pilot programs will include data comparing 
students’ performance in classroom activity with the scores 
experimental measures yield. This information, complemented by 
observation of and interviews with students, will provide 
insights on the apparent validity of experimental measures and 
the effects they have on individual learners. If pilot programs 
reflect a mission of experimentation, discovery, and 
modification, then evaluation will be shaped by instruction in 
the tradition of process learning. Rather than imposing 
arbitrary, external objectives, evaluation will grow out of 
insights on cognitive activity and development. 

The findings of small-scale programs will be more informative 
than generalizable , yet they are essential if attempts to reform 
large-scale evaluation are to continue. A strong network of 
communication through which insights on assessment are shared is 
a necessary complement for sustaining the effort. Since 
research findings must be accessible to both teachers and 
researchers, given the cognitive focus and practical aims of the 
research agenda, they should not be limited to publications on 
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education in general or to technical psychology journals. With 
new insights on testing focusing on the learning context and the 
content areas, national and regional organizations of teachers 
in the disciplines offer a means for broadening the scope of the 
research effort. Exchanges among professionals will point to 
the significance of the research agenda, ways to apply its 
findings, and any modifications which appear appropriate. Only 
through a strong network of communication reaching both 
theoreticians and educators can a movement to alter evaluation 
gain momentum. 

A process approach to learning illustrates theory in 
practice. Process learning narrows the gap separating research 
efforts and the classroom experience. While a product approach 
reinforces the separation of these (behavior constituting a 
research concern and the transmission of information an 
educational objective), process learning does not. It attempts 
to reconcile theories about what should occur in education with 
the accounts of what actually transpires in the classroom. 



CONCLUSION 

With process instruction gaining momentum in individual 
classrooms, the time has come to organize the standards 
underlying this instructional approach to facilitate the 
large-scale implementation of suitable assessment measures. 
While process evaluation might appear difficult to apply on a 
large scale, given its focus on individuals and activity, a 
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review of the literature indicates that an effort to achieve 
this new vision of assessment has been developing for two 
decades. Introducing process learning, an approach contemporary 
society has demanded, research and education have prepared us 
for a new way of speaking about learning and student progress. 
This shift in the focus of evaluation represents a critical, 
necessary change in education, given the problems threatening it 
today. The large-scale implementation of process evaluation 
will unify discussions of students, drawing from a literature on 
growth and development. Aligning educational experiences, the 
social context, and cognitive research, this shift in the focus 
of assessment will reconcile instruction and evaluation, making 
education an integral event, and it will focus on individual 
learners, offering them opportunities for growth. 
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